Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Badger] Refactor approvals to use badger batch updates #6466

Draft
wants to merge 9 commits into
base: leo/db-ops
Choose a base branch
from

Conversation

zhangchiqing
Copy link
Member

@zhangchiqing zhangchiqing commented Sep 14, 2024

Please review #6465 first.

The first attempt of the refactor refactored the approvals from using badger transaction to badger batch updates, allowing us to transit to pebble based implementation easier. However, it would still require keeping two implementations of the operations functions (storage/badger/operation/approvals.go and storage/pebble/operation/approvals.go) and corresponding storage modules.

This PR builds on top of the universal database operations, and implements only one version of database operations and store module. Easier to maintain than having two versions. Also makes it easy to switch to pebble version (explained in review comments).

@codecov-commenter
Copy link

codecov-commenter commented Sep 14, 2024

Codecov Report

Attention: Patch coverage is 51.43954% with 253 lines in your changes missing coverage. Please review.

Project coverage is 41.47%. Comparing base (2a443de) to head (299f064).

Files with missing lines Patch % Lines
utils/unittest/unittest.go 0.00% 41 Missing ⚠️
storage/operation/pebbleimpl/iterator.go 0.00% 29 Missing ⚠️
storage/operation/reads.go 70.52% 15 Missing and 13 partials ⚠️
storage/operation/dbtest/helper.go 25.00% 27 Missing ⚠️
storage/operation/badgerimpl/iterator.go 0.00% 26 Missing ⚠️
storage/store/cache.go 65.15% 22 Missing and 1 partial ⚠️
storage/operation/codec.go 27.27% 16 Missing ⚠️
storage/operation/badgerimpl/writer.go 63.88% 13 Missing ⚠️
storage/operation/approvals.go 0.00% 8 Missing ⚠️
storage/operation/pebbleimpl/writer.go 76.66% 7 Missing ⚠️
... and 9 more
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6466      +/-   ##
==========================================
+ Coverage   41.44%   41.47%   +0.03%     
==========================================
  Files        2026     2042      +16     
  Lines      144629   145107     +478     
==========================================
+ Hits        59937    60190     +253     
- Misses      78492    78699     +207     
- Partials     6200     6218      +18     
Flag Coverage Δ
unittests 41.47% <51.43%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@@ -201,7 +203,7 @@ func (v *VerificationNodeBuilder) LoadComponentsAndModules() {
vmCtx := fvm.NewContext(fvmOptions...)

chunkVerifier := chunks.NewChunkVerifier(vm, vmCtx, node.Logger)
approvalStorage := badger.NewResultApprovals(node.Metrics.Cache, node.DB)
approvalStorage := store.NewResultApprovals(node.Metrics.Cache, badgerimpl.ToDB(node.DB))
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switching from badger to pebble in the future would just be changing badgerimpl to pebbleimpl here.

@@ -1,31 +0,0 @@
package operation
Copy link
Member Author

@zhangchiqing zhangchiqing Sep 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is moved to storage/operation/approvals.go, however git is not able to recognize it as a git file rename, still consider it as a deletion here.

@@ -1,35 +1,33 @@
package badger
package store
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub correctly recognized the changes are actually on top of a file renaming, much easier to view the actual changes. And there is no pebble version to review, because this is a universal one.

@AlexHentschel AlexHentschel self-requested a review October 22, 2024 17:33
@zhangchiqing zhangchiqing force-pushed the leo/db-ops branch 7 times, most recently from 77b44d1 to 17972e0 Compare November 2, 2024 00:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants